Adjusted Viterbi training for hidden Markov models

To estimate the emission parameters in hidden Markov models one commonly uses the EM algorithm or its variation. Our primary motivation, however, is the Philips speech recognition system wherein the EM algorithm is replaced by the Viterbi training algorithm. Viterbi training is faster and computationally less involved than EM, but it is also biased and need not even be consistent. We propose an alternative to the Viterbi training -- adjusted Viterbi training -- that has the same order of computational complexity as Viterbi training but gives more accurate estimators. Elsewhere, we studied the adjusted Viterbi training for a special case of mixtures, supporting the theory by simulations. This paper proves the adjusted Viterbi training to be also possible for more general hidden Markov models.Comment: 45 pages, 2 figure

A generalized risk approach to path inference based on hidden Markov models

Motivated by the unceasing interest in hidden Markov models (HMMs), this paper re-examines hidden path inference in these models, using primarily a risk-based framework. While the most common maximum a posteriori (MAP), or Viterbi, path estimator and the minimum error, or Posterior Decoder (PD), have long been around, other path estimators, or decoders, have been either only hinted at or applied more recently and in dedicated applications generally unfamiliar to the statistical learning community. Over a decade ago, however, a family of algorithmically defined decoders aiming to hybridize the two standard ones was proposed (Brushe et al., 1998). The present paper gives a careful analysis of this hybridization approach, identifies several problems and issues with it and other previously proposed approaches, and proposes practical resolutions of those. Furthermore, simple modifications of the classical criteria for hidden path recognition are shown to lead to a new class of decoders. Dynamic programming algorithms to compute these decoders in the usual forward-backward manner are presented. A particularly interesting subclass of such estimators can be also viewed as hybrids of the MAP and PD estimators. Similar to previously proposed MAP-PD hybrids, the new class is parameterized by a small number of tunable parameters. Unlike their algorithmic predecessors, the new risk-based decoders are more clearly interpretable, and, most importantly, work "out of the box" in practice, which is demonstrated on some real bioinformatics tasks and data. Some further generalizations and applications are discussed in conclusion.Comment: Section 5: corrected denominators of the scaled beta variables (pp. 27-30), => corrections in claims 1, 3, Prop. 12, bottom of Table 1. Decoder (49), Corol. 14 are generalized to handle 0 probabilities. Notation is more closely aligned with (Bishop, 2006). Details are inserted in eqn-s (43); the positivity assumption in Prop. 11 is explicit. Fixed typing errors in equation (41), Example

A constructive proof of the existence of Viterbi processes

Since the early days of digital communication, hidden Markov models (HMMs) have now been also routinely used in speech recognition, processing of natural languages, images, and in bioinformatics. In an HMM $(X_i,Y_i)_{i\ge 1}$, observations $X_1,X_2,...$ are assumed to be conditionally independent given an ``explanatory'' Markov process $Y_1,Y_2,...$, which itself is not observed; moreover, the conditional distribution of $X_i$ depends solely on $Y_i$. Central to the theory and applications of HMM is the Viterbi algorithm to find {\em a maximum a posteriori} (MAP) estimate $q_{1:n}=(q_1,q_2,...,q_n)$ of $Y_{1:n}$ given observed data $x_{1:n}$. Maximum {\em a posteriori} paths are also known as Viterbi paths or alignments. Recently, attempts have been made to study the behavior of Viterbi alignments when $n\to \infty$. Thus, it has been shown that in some special cases a well-defined limiting Viterbi alignment exists. While innovative, these attempts have relied on rather strong assumptions and involved proofs which are existential. This work proves the existence of infinite Viterbi alignments in a more constructive manner and for a very general class of HMMs.Comment: Submitted to the IEEE Transactions on Information Theory, focuses on the proofs of the results presented in arXiv:0709.2317, and arXiv:0803.239

Infinite Viterbi alignments in the two state hidden Markov models

Since the early days of digital communication, Hidden Markov Models (HMMs) have now been routinely used in speech recognition, processing of natural languages, images, and in bioinformatics. An HMM $(X_i,Y_i)_{i\ge 1}$ assumes observations $X_1,X_2,...$ to be conditionally independent given an "explanotary" Markov process $Y_1,Y_2,...$, which itself is not observed; moreover, the conditional distribution of $X_i$ depends solely on $Y_i$. Central to the theory and applications of HMM is the Viterbi algorithm to find {\em a maximum a posteriori} estimate $q_{1:n}=(q_1,q_2,...,q_n)$ of $Y_{1:n}$ given the observed data $x_{1:n}$. Maximum {\em a posteriori} paths are also called Viterbi paths or alignments. Recently, attempts have been made to study the behavior of Viterbi alignments of HMMs with two hidden states when $n$ tends to infinity. It has indeed been shown that in some special cases a well-defined limiting Viterbi alignment exists. While innovative, these attempts have relied on rather strong assumptions. This work proves the existence of infinite Viterbi alignments for virtually any HMM with two hidden states.Comment: Several minor changes and corrections have been made in the arguments as suggested by anonymous reviewers, which should hopefully improve readability. Abstract has been adde

Regularisation, interpolation and visualisation of diffusion tensor images using non-Euclidean statistics.

Practical statistical analysis of diffusion tensor images is considered, and we focus primarily on methods that use metrics based on Euclidean distances between powers of diffusion tensors. First we describe a family of anisotropy measures based on a scale invariant power-Euclidean metric, which are useful for visualisation. Some properties of the measures are derived and practical considerations are discussed, with some examples. Second we discuss weighted Procrustes methods for diffusion tensor interpolation and smoothing, and we compare methods based on different metrics on a set of examples as well as analytically. We establish a key relationship between the principal-square-root-Euclidean metric and the size-and-shape Procrustes metric on the space of symmetric positive semi-definite tensors. We explain, both analytically and by experiments, why the size-and-shape Procrustes metric may be preferred in practical tasks of interpolation, extrapolation, and smoothing, especially when observed tensors are degenerate or when a moderate degree of tensor swelling is desirable. Third we introduce regularisation methodology, which is demonstrated to be useful for highlighting features of prior interest and potentially for segmentation. Finally, we compare several metrics in a dataset of human brain diffusion-weighted MRI, and point out similarities between several of the non-Euclidean metrics but important differences with the commonly used Euclidean metric

Intra-operative spectroscopic assessment of surgical margins during breast conserving surgery

Background: In over 20% of breast conserving operations, postoperative pathological assessment of the excised tissue reveals positive margins, requiring additional surgery. Current techniques for intra-operative assessment of tumor margins are insufficient in accuracy or resolution to reliably detect small tumors. There is a distinct need for a fast technique to accurately identify tumors smaller than 1 mm2 in large tissue surfaces within 30 min. Methods: Multi-modal spectral histopathology (MSH), a multimodal imaging technique combining tissue auto-fluorescence and Raman spectroscopy was used to detect microscopic residual tumor at the surface of the excised breast tissue. New algorithms were developed to optimally utilize auto-fluorescence images to guide Raman measurements and achieve the required detection accuracy over large tissue surfaces (up to 4 × 6.5 cm2). Algorithms were trained on 91 breast tissue samples from 65 patients. Results: Independent tests on 121 samples from 107 patients - including 51 fresh, whole excision specimens - detected breast carcinoma on the tissue surface with 95% sensitivity and 82% specificity. One surface of each uncut excision specimen was measured in 12–24 min. The combination of high spatial-resolution auto-fluorescence with specific diagnosis by Raman spectroscopy allows reliable detection even for invasive carcinoma or ductal carcinoma in situ smaller than 1 mm2. Conclusions: This study provides evidence that this multimodal approach could provide an objective tool for intra-operative assessment of breast conserving surgery margins, reducing the risk for unnecessary second operations